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Abstract 

This  paper  discusses  a  critical  gap  in  the  U.S.  Navy  acquisition  process.  This  gap  is  caused 
by  the  absence  of  workforce  alignment  metrics  and  metadata  algorithms  in  two  areas:  (1)  As 
applied  to  determining  estimated  cost  per  work  breakdown  element  as  part  of  the  bid  analysis 
process;  and  (2)  after  the  award  is  granted,  as  part  of  the  task  management  functions  to 
ensure  expectations  associated  with  the  original  estimate  can  be  reliably  fulfilled.  In  this 
paper,  the  integration  of  workforce  alignment  metrics  that  are  processed  by  a  statistically- 
based,  metadata-driven  algorithm  is  referred  to  as  the  6-3-5  Method.  The  6-3-5  Method  was 
specifically  engineered  using  results  from  numerous  case  studies  from  NAVSEA-  and  NATO- 
based  programs  focused  on  creating  adequate  cost  control  measures  in  support  of  the 
acquisition  process.  Based  on  these  case  studies,  it  is  shown  that  the  Department  of  the 
Navy’s  acquisition  process  has  a  significant  gap  in  its  ability  to  adequately  provide  cost 
control.  The  case  studies  demonstrate  how  the  6-3-5  Method  fills  this  gap,  ensuring  the  future 
financial  health  and  competitive  status  of  the  U.S.  Navy  to  adequately  address  emerging 
threats  to  U.S.  national  defense. 

Introduction 

The  need  for  Navy  leadership  to  evolve  its  current  cost  control  solution  has  become 
even  more  pressing  with  current  discussions  about  financial  uncertainty,  talent  management 
and  better  use  of  workforce  innovation.  For  example,  the  cover  of  the  August  2015  issue  of 
National  Defense  magazine  reads,  “Pressure  Mounts  to  Fund  Ohio  Replacement.”  At  the 
Navy  League’s  Sea-Air-Space  (SAS)  2015  Symposium,  Admiral  James  A.  Winnefeld,  Jr., 
Vice  Chairman  of  the  Joint  Chiefs  of  Staff,  during  his  banquet  address,  stated  that  “budget 
cuts  are  causing  significant  uncertainty”  (Winnefeld,  2015).  Defense  News’  August  2015 
interview  with  Brad  Carson,  acting  Department  of  Defense  (DoD)  personnel  and  readiness 
chief,  focused  on  the  need  to  take  advantage  of  the  unique  talents  of  military  and  civil 
service  employees  (Carson,  2015). 

Secretary  of  the  Navy  (SECNAV)  Ray  Mabus’  recent  speech  at  the  same  Navy 
League  event  stated  a  need  for  “innovation”  as  an  inherent  part  of  Navy  culture  to  combat 
these  troubling  economic  times  more  effectively  (Mabus,  2015).  These  insights  from  DoD 
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and  Department  of  the  Navy  (DoN)  leadership  are  supported  by  research  findings  led  by 
Google,  Inc.  These  finding  reveal  the  need  for  an  organization  to  create  a  culture  that  uses 
talent  more  effectively  through  collaborative  processes  that  promote  workforce  innovation 
(Duhigg,  2016). 

This  paper  introduces  the  6-3-5  Method  that  integrates  metadata  statistics  and 
assessment  heuristics  to  promote  better  use  of  “big”  data  and  workforce  collaboration  as 
applied  to  defense  acquisition  and  related  management  activities.  The  6-3-5  Method 
consists  of  a  metadata-based  algorithm,  statistical  analytics,  a  heuristic  methodology, 
various  measurements,  and  a  visual  approach  to  display  results.  Three  case  studies, 
demonstrating  the  use  of  the  6-3-5  Method,  add  proof  to  the  need  to  address  the  challenges 
described  by  DoD  and  DoN  leadership,  as  well  as  validate  Google  research  results 
regarding  the  benefits  from  collaboration  processes  that  promote  workforce  innovation. 

The  case  studies  demonstrate  how  the  6-3-5  Method  uses  performance  data  to  align 
more  effectively  workforce  talent  to  goals,  reduce  cost  variances  and  improve  cost  control. 
The  methodology  provides  that  ability  to  assess  strengths  and  weaknesses  in  the 
architectural  framework  of  an  organization’s  cost  control  approach.  Also,  it  provides  specific 
recommendation  solutions  and  clear  direction  to  fill  identified  gaps  or  weaknesses  in  the 
framework  to  increase  the  likelihood  of  successful  outcomes  (i.e.,  actuals  equaling  estimates 
without  compromise),  whether  in  the  form  of  Task  Planning  Sheet  (TPS)  deliverables 
assigned  to  government  civil  service  employees  or  Ships  Work  List  Item  Number  (SWLIN) 
deliverables  for  overhaul/new  construction  performed  by  prime  contractors/shipbuilders. 

The  paper  presents  the  6-3-5  Method  and  related  case  studies  in  the  following  order: 

•  Overview  of  the  6-3-5  Method 

•  How  the  6-3-5  Method  supports  the  acquisition  process  based  on  a  Program 
Executive  Office  (PEO)  Carriers  Case  Study 

•  How  the  6-3-5  Method  supports  workforce  management  based  on  a  NATO 
Program  Case  Study 

•  How  the  6-3-5  Method  supports  the  workforce  management  based  on  a 
Naval  Warfare  Center  Case  Study 

The  first  case  study  involving  PEO  Carriers  provides  examples  of  two  significant  cost 
control  issues  that  are  addressed  using  the  6-3-5  Method.  This  case  study  exemplifies  how 
the  6-3-5  Method  can  be  applied  to  any  DoD  request  for  proposal  (RFP)  process  involving 
the  review  of  bids  for  cost  estimation  accuracy.  In  this  case  study,  the  6-3-5  Method 
emphasizes  a  better  use  of  the  cost  control  data  that  is  required  by  programs  that  are 
required  contractually  to  provide  earned  value  analysis. 

The  mathematics  is  based  on  Van  Trees’s  work  on  “Detection,  Estimation  and 
Modulation  Theory”  (Van  Trees,  2001).  The  application  is  based  on  viewing  data  to  identify 
an  average  performance  range,  where  reliable  performance  reduces  cost  variances.  The 
algorithm  implementing  the  6-3-5  Method  processes  the  data,  determines  the 
highest/maximum  likelihood  that  the  average  performance  is  true  and  not  a  false  positive. 
The  algorithm’s  mathematical  basis  has  been  peer  reviewed  and  supported  by  California 
State  University  faculty.  This  analysis  provides  an  accurate  cost  analysis  of  a  bid  or  financial 
estimate.  This  paper  introduces  how  detection  and  estimation  mathematics  can  be  applied 
to  metadata  (data  about  data)  and  algorithm  processing  based  on  an  Average  Performance 
Range  and  Index  (APRI)  table,  as  described  in  Figure  1 . 
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Figure  1.  Average  Performance  Range  and  (Lambda)  Index  Structure  Used  by  6-3- 

5  Method 

There  are  two  challenges  for  an  acquisition  management  team  that  are  discussed  in 
this  paper.  The  first  challenge  involves  bid  assessment. 

Bid  assessment  is  challenged  not  only  when  assessing  new  technology  costs,  but 
also  when  applying  it  to  upgrades  to  existing  technology  already  deployed  within  the  fleet.  In 
this  paper,  past  performance  analysis  of  a  bid  is  turned  into  metadata  and  algorithmically 
analyzed  using  an  APRI  table.  The  analysis  results  in  determining  statistical  confidence 
intervals,  more  conventionally  discussed  as  statistical  confidence,  as  a  more  effective 
means  to  determine  the  accuracy  of  a  bid  or  estimation  cost.  Specifically,  a  PEO  Carriers 
case  study  is  reviewed  involving  an  aircraft  carrier  overhaul,  where  SWLINs  within  the  bid 
are  statistically  analyzed  to  determine  the  likelihood  of  issues  not  meeting  contractual  cost 
control  requirements  in  compliance  with  the  Nunn-McCurdy  Amendment  (Nunn-McCurdy 
Act,  1983).  In  this  case  study,  the  6-3-5  Method  provides  analysis  to  the  acquisition 
management  team  before  bid  acceptance  based  on  past  performance  history  using  the 
same  “actuals”  as  compared  to  estimated  data  that  supported  earned  value  calculations. 
(Albeit  outside  the  scope  of  first  case  study:  the  6-3-5  Method  also  can  provide  analysis 
using  heuristics  that  can  later  be  verified  with  performance  data.) 

The  first  case  study  demonstrates  how  applying  a  feature  of  the  6-3-5  Method 
provides  an  accurate  gap  analysis  for  each  SWLIN  cost  estimate  described  within  the  bid, 
where  each  line  item  is  described  by  a  statistical  confidence  interval,  having  both  an  upper 
bound  and  lower  bound.  This  interval  can  be  mapped  to  the  contractual  cost  control 
requirements  needed  to  be  satisfied  by  the  offering  prime  contractor.  The  6-3-5  Method  also 
provides  a  structured  approach  to  ensure  the  offeror  is  responding  adequately  to  issues  to 
ensure  the  proper  due  diligence  necessary  to  fill  those  identified  gaps  before  bid 
acceptance. 

The  second  challenge  for  an  acquisition  management  team  involves  the  need  to 
ensure  that  there  is  adequate  due  diligence  by  the  offering  prime  contractor  to  resolve 
issues  identified  as  statistically  not  meeting  cost  control  contractual  requirements.  This  due 
diligence  is  essential  to  complete  before  the  contract  award  is  granted  to  ensure  minimal 
cost  variance  during  implementation.  The  first  case  study  emphasizes  this  key  concept. 

Providing  a  structured  due  diligence  approach  that  can  be  statistically  analyzed  in 
terms  of  confidence  is  a  necessary  government  procedure  that  can  no  longer  be  left,  “at 
best,”  to  ad  hoc  processes.  All  too  often,  prime  contractors  increase  their  profits  when 
Engineering  Change  Orders  (ECOs)  are  generated.  Each  ECO  causes  a  decrease  in  cost 
control  for  the  government  acquisition  management  team,  which  can  result  in  significant 
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cost  overruns  and  schedule  delays.  The  size  of  the  cost  the  government  must  pay  for  an 
ECO  is  proportional  to  the  size  of  the  profit  gained  by  the  contractor  and  inversely 
proportional  to  decrease  in  cost  control  for  the  program.  The  only  exception  is  with  Fixed 
Price  contracts,  which  have  other  issues  this  paper  addresses  in  terms  of  quality 
compromises  needed  by  the  government  versus  contractor  profit  margin  targets  (FAR, 

2016). 

Applying  the  6-3-5  Method  allows  the  acquisition  management  team  to  determine 
objectively  the  rigor  of  due  diligence  that  was  applied  to  minimize  potential  impact  to  the 
government,  thereby  reducing  the  use  of  ECOs  or  minimizing  their  effects  on  cost  variance 
and  potential  schedule  delays.  This  first  case  study  shows  how  this  type  of  rigorous 
statistical  analysis  fills  a  critical  gap  in  DoN’s  cost  control  solution,  and  should  be  considered 
as  the  first  step  to  manage  acquisition  costs  adequately. 

The  second  case  study  demonstrates  the  required  cost  control  objectives  (CCOs) 
within  a  program  and  the  procedural  management  steps  needed  to  achieve  adequate  due 
diligence  using  the  6-3-5  Method.  Specifically,  this  case  study  involves  a  NATO-sponsored 
development  and  acquisition  program.  It  outlines  key  aspects  of  the  6-3-5  Method  in  terms 
of  workforce  management,  alignment  and  innovation.  This  NATO  case  study  views  use  of 
the  6-3-5  Method  from  the  prime  contractor’s  point  of  view,  focusing  on  risk  analysis  of  the 
project  plan  and  validation  of  an  effective  mitigation  strategy  to  reduce  ECOs.  It  introduces 
the  use  of  workforce  alignment  metrics  and  a  metadata-based  algorithm,  compliant  with  the 
6-3-5  Method,  to  promote  and  enable  workforce  innovation  as  an  effective  solution  to 
addressing  unknowns  during  task  assignment  and  implementation.  The  workforce  alignment 
metrics  and  metadata-based  algorithm  provide  the  acquisition  team  with  a  measurable, 
objective  way  to  ensure  adequate  due  diligence  is  being  performed  before  an  ECO  needs  to 
be  generated  or  alternatively,  a  mitigation  strategy  is  implemented. 

Use  of  workforce  alignment  metrics  and  the  related  metadata-based  algorithm 
becomes  the  focus  of  the  third  and  final  case  study  involving  deliverables  listed  in  Task 
Planning  Sheets  and  assigned  to  a  branch  within  a  NAVSEA  naval  warfare  center  of 
excellence.  The  case  study  emphasizes  the  need  for  workforce  innovation  at  the  task 
execution  level,  when  daily  challenges  by  the  workforce  are  encountered  and  their  ability  to 
succeed  relates  directly  to  the  quality  of  service  provided.  This  type  of  government 
environment  in  which  a  branch  of  civil  service  employees  must  do  assigned  work  can  be 
equated  to  a  firm  fixed-price  contract  between  the  branch  and  related  PEO  to  provide  the 
agreed  upon  service  and  meet  the  expectations  described  within  the  Task  Planning  Sheet. 

Methodology  Overview 

The  6-3-5  Method  consists  of  a  metadata-based  algorithm,  statistical  analytics,  a 
heuristic  methodology,  various  measurements,  and  a  visual  approach  to  display  results. 

Fundamentally,  there  are  only  two  types  of  metrics,  a  priori  and  a  posteriori,  that 
apply  to  decision-making.  Using  the  6-3-5  Method,  these  metrics  are  used  to  create  six 
metadata  tags.  A  metadata  tag  provides  intelligence  in  terms  of  what  a  data  value  means. 
The  6-3-5  Method  requires  a  total  of  six  metadata  tags  to  support  the  acquisition  effort’s  task 
management  process,  from  estimation  to  product/service  delivery.  All  six  metadata  tags 
ensure  leadership  is  making  informed  decisions  that  have  the  highest  likelihood  of  having 
successful  outcomes  and  maintaining  cost  control.  Each  metadata  tag  is  created  from  either 
an  a  priori  or  a  posteriori  metric  type.  The  metadata  tagging  for  a  priori  or  a  posteriori  metric 
types  are  described  in  Figure  2. 
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Figure  2.  A  Priori  anti  A  Posteriori  Metadata  Tagging  Used  by  the  6-3-5  Method 

The  first  three  metadata  tags  (MT)  support  a  priori  (proactive)  measurements.  These 
three  tags  focus  on  providing  intelligence  to  uninterpreted  raw  data  values  in  order  to  (1) 
prevent  forecasted  issues,  (2)  eliminate  impact  of  existing  issues,  or  (3)  when  prevention 
and  elimination  cannot  occur,  minimize  cost  variance  factors: 

MT-1.  Statistically-Based  Lessons  Learned  Metadata:  This  is  metadata  that 
supports  the  analytics  to  do  a  statistically-based  gap  analysis.  This  analysis 
determines  whether  there  are  any  lessons  learned  before  proceeding  forward 
in  implementing  an  action.  That  action  could  be  accepting  a  prime 
contractor’s  bid  or  allowing  tasks  to  be  implemented,  during  which  time  and 
money  are  consumed.  The  6-3-5  Method  uses  past  performance 
measurements  or  characteristically  similar  data  (determined  by  an  algorithm’s 
process  flow  described  in  Figure  6)  mapped  to  an  APRI  table  via  MT-4  that 
are  converted  into  higher  level  tagging  that  support  statistically-based  lessons 
learned  metadata.  These  metadata  tags  are  then  analyzed  mathematically  to 
perform  gap  analysis  on  the  statistical  likelihood  of  success  in  meeting  cost 
control  expectations.  Low  statistical  confidence  identifies  potential  for  cost 
overruns.  If  the  statistical  confidence  is  in  the  red  zone  of  the  APRI  table, 
significant  cost  overruns  with  statistical  confidence  are  forecast.  This 
metadata  measurement  forecast  provides  management  with  actionable  data 
that  can  be  used  to  proactively  prevent/mitigate  cost  variances  using  the 
proactive  due  diligence  (MT-3). 

MT-2.  Proactive  Assessment  Metadata:  This  is  metadata  that  supports  the 
analytics  to  do  a  heuristically-based  gap  analysis.  The  workforce  self- 
assessment  answers  are  mapped  to  an  APRI  table  that  is  converted  into 
metadata.  The  metadata  is  analyzed  heuristically  to  provide  a  gap  analysis  of 
potential  cost  overrun  issues,  independent  of  MT-1  results,  again  based  on 
the  likelihood  of  successfully  meeting  cost  control  expectations.  When 
performance  data  is  mapped  to  the  assessment  metadata  tags  via  MT-4,  the 
measurements  are,  once  again,  tagged  respectively  with  a  statistical 
confidence  of  the  gap  identified  and  each's  impact  to  cost.  This  metadata 
assessment  measurement  provides  actionable  data  for  management  to  use 
to  proactively  prevent/mitigate  cost  variances  using  the  proactive  due 
diligence,  addressed  by  MT-3. 
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MT-3.  Proactive  Due  Diligence  Metadata:  This  is  metadata  that  validates 
“best”  solution  selected  for  resolving  an  issue.  When  a  priori  metadata  tags, 
either  from  MT-1  (statistics-based)  or  MT-2  (heuristics-based),  indicate  a  gap 
in  achieving  reliable  results,  this  issue  initializes  the  due  diligence  process 
when  following  the  6-3-5  Method.  Due  diligence  metadata  tags  are  used  to 
ensure  adequate  rigor  is  achieved  in  resolving  these  cost  control  issues  and 
preventing/minimizing  cost  variance.  The  metadata  from  either  MT-1  or  MT-2 
is  inserted  into  a  problem-solving  format  in  accordance  with  the  6-3-5  Method 
Problem  Solving  Collaboration  Approach  later  described  (Case  Study  2). 
Using  these  constructs  results  in  solutions  that  are  mapped  into  an  APRI 
table  and  converted  into  due  diligence  metadata. 

The  due  diligence  metadata  determines  the  degree  to  which  the  solution  fills  the 
gaps  originally  identified.  If  the  gaps  are  not  adequately  filled  (i.e.,  complete  prevention  of 
the  cost  variance  issue),  then  the  due  diligence  metadata  highlights  those  core  areas  of 
concern,  where  a  mitigation  strategy  is  identified  to  minimize  impact.  If  gaps  are  filled  as 
validated  by  the  metadata  tags,  then  management  also  receives  validation  from  the 
measurement  via  a  recommendation  as  to  the  best  corrective  action  to  preclude/mitigate 
cost  variance.  When  past  performance  data  becomes  available,  this  data  gets  mapped  to 
the  due  diligence  metadata  using  the  APRI  table  via  MT-4.  The  result  provides  a  higher  level 
tag  that  determines  statistical  confidence  of  the  corrective  action,  specifically  forecasting  the 
likelihood  of  success  when  implementing  the  determined  solution. 

The  final  three  MTs,  based  on  a  posteriori  (reactive)  metric  types,  focus  on  providing 
intelligence  to  untagged  (or  “raw”)  data  values  in  order  to  recover  or  minimize  factors  that 
have  been  measured  as  having  impact  (i.e.,  increasing  cost  variance  during  or  after 
implementation). 

MT-4.  Performance  Tracking  Metadata:  This  is  metadata  created  from 
performance  tracking  measurements.  Specifically,  these  are  estimates 
compared  to  actuals  regarding  Full  Time  Equivalent  (FTE)  hours  and 
schedule  start  and  finish  dates.  Performance  data  collected  is  mapped  into 
an  APRI  table,  where  the  results  are  converted  into  metadata  tags  that 
support  both  heuristic-  and  statistic-based  analysis  used  by  MT-1,  MT-2,  MT- 
3,  MT-5  and  MT-6.  In  support  of  the  DoN’s  cost  control  decision-making,  use 
of  intelligently  tagged  performance  tracking  measurements  that  can  lead  to 
higher  level  tagging  constructs  is  recommended.  Even  earned  value 
management  techniques  have  significant  limitations  in  helping  decision 
makers  know  core  issues  and  “best”  corrective  actions  to  provide  highest 
probability  of  success.  All  the  other  MTs  are  statistically  dependent  on  this 
MT-4’s  APRI  tagging,  which  can  be  translated  to  core  issues  via  tags,  to 
provide  insights  into  “best”  corrective  actions  and  reveal  the  statistical 
confidence,  again  as  metatags,  of  a  potential  solution  for  success.  Metadata 
tables  displaying  APRI  tagging  and  statistical  confidence  are  shown  in  the 
NAVSEA  case  studies  (Case  Studies  1  and  3). 

MT-5.  Reactive  Due  Diligence  Metadata:  This  is  metadata  that  validates 
“best”  solution  selected  for  resolving  an  issue.  When  a  posteriori  metadata 
tags,  either  from  MT-4  (performance  data)  or  MT-6  (lessons  learned  data), 
indicate  an  issue,  the  due  diligence  process  is  initiated  when  following  the  6- 
3-5  Method.  It  is  similar  to  MT-3,  with  the  exception  that  this  is  a  reactive  or 
after  the  fact.  Due  diligence  metadata  tags  are  used  to  ensure  adequate  rigor 
is  achieved  in  resolving  these  cost  control  issues  and  recovering/minimizing 


ACQUISITION  RESEARCH  PROGRAM: 
CREATING  SYNERGY  FOR  INFORMED  CHANGE 


-297- 


cost  variance.  The  metadata  from  MT-4  or  MT-6  is  inserted  into  a  problem¬ 
solving  format  in  accordance  with  the  6-3-5  Method’s  Problem  Solving 
Collaboration  Approach.  Using  these  constructs  results  in  solutions  that  are 
mapped  into  an  APRI  table  and  converted  into  due  diligence  metadata. 

The  due  diligence  metadata  determines  the  degree  to  which  the  solution  fills 
the  gaps  originally  identified.  If  the  gaps  are  not  adequately  filled,  then 
complete  recovery  of  the  cost  variance  issue,  the  due  diligence  metadata 
highlights  core  areas  of  concern,  where  a  mitigation  strategy  is  identified  to 
minimize  impact.  If  gaps  are  filled  as  validated  by  the  metadata  tags,  then 
management  also  receives  validation  from  the  measurement  as  to  best 
corrective  action  to  recover  cost  variance  impact.  Using  MT-4,  a  statistical 
confidence  can  forecast  the  likelihood  of  success  for  the  identified  corrective 
action. 

MT-6.  On-the-Job  Lessons  Learned  Metadata:  This  is  metadata  that  supports 
the  analytics  to  do  a  heuristically-based  gap  analysis.  This  tag  can  be 
translated  to  organizational  learning,  both  heuristically  and  statistically.  The 
workforce  lessons  learned  assessments  are  mapped  to  an  APRI  table  that  is 
converted  into  metadata.  The  metadata  is  analyzed  heuristically  or 
statistically,  based  on  MT-4,  to  provide  objective  lessons  learned.  Because  of 
the  metadata  tagging  constructs,  the  lessons  learned  can  be  translated  for 
use  throughout  the  organization  and  is  not  limited  to  the  project  or  team 
generating  the  learning.  The  on-the-job  lessons  learned  metadata  focuses  on 
“what  worked”  and  “what  didn’t  work”  with  regard  to  the  (1)  customer,  (2) 
organization,  (3)  teams,  and  (4)  individuals  performing  the  work.  This 
metadata  is  also  valuable  in  identifying  a  need  for  solutions  to  proactively 
prevent  future  cost  control  issues  throughout  various  projects  and  activities 
within  the  organization.  The  main  difference  between  MT-6  and  MT-3  is  that 
MT-6  involves  the  archiving  of  lessons  learned  for  others  to  use  at  some 
future  date,  where  MT-3  is  lessons  learned  to  address  an  immediate  tactical 
need. 


on  A  Amen  Mttria 


■>  MT-l 


.>  MT  J 
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*»  MT'l 


Figure  3.  Metadata  Tags  From  APRI  Mapping  Examples 
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Figure  3  graphically  describes  how  MTs  use  APRI  mapping.  The  six  MTs  require 
context  for  use.  The  three  Cost  Control  Objectives  (CCOs)  provide  a  context  in  which  the 
previous  six  metadata  tags  are  used.  Providing  this  context  ensures  management  knows 
how  to  deploy  and  maintain  the  previously  described  six  metadata  tags  to  optimize  spending 
control  and  reduce  cost  variance. 

CCO-1 — Set  and  manage  customer/business  expectations:  The  first  CCO 
ensures  expectations  are  sufficiently  defined  based  on  seven  categories.  The 
first  five  categories  deal  with  productivity  needs  and  the  last  two  focus  on 
efficiency  needs  for  acquisition  programs  and  operational  workflow 
management.  The  categories  are:  (1)  Requirements,  (2)  Quality,  (3)  Process, 
(4)  Technology,  (5)  Culture,  (6)  Cost  (Including  Workforce  Allocation  Hours), 
and  (7)  Schedule/Timeline 

CCO-2 — Reliably  achieve  those  expectations:  The  second  CCO  ensures  that 
those  defined  expectations  are  reliably  achieved,  without  compromise.  Steps 
within  this  CCO  include  gap  analysis  and  due  diligence  before,  during  and 
after  implementation  to  ensure  cost  variances  are  minimized  during  the  life  of 
the  project. 

CCO-3 — Learn  to  continually  do  better:  The  third  and  final  CCO  focuses  on 
Continuous  Process  Improvement  to  ensure  the  workforce  is  continually 
learning  to  be  better  at  providing  reliable,  quality  services/products,  including 
how  to  better  collaborate  and  innovate  when  overcoming  challenges.  There 
are  four  categories  that  represent  accumulated  lessons  learned.  Those 
categories  are:  (1)  customer,  (2)  business/organization,  (3)  team,  and  (4) 
individuals  associated  with  the  quality  and  reliability  of  the  work  performed. 

With  the  three  CCOs,  a  context  for  using  the  six  metadata  tags  is  described.  Yet, 
MT-3  and  MT-5  require  the  use  of  the  6-3-5  Method’s  problem-solving  constructs.  The  6-3-5 
Method’s  problem-solving  approach  is  in  the  form  of  five  Due  Diligence  Steps  (DDS)  to 
metrically  ensure  rigor  in  handling  the  issues  identified  as  cost  control  gaps.  The  sequence 
of  how  these  steps  are  described  is  in  Figure  4.  These  steps  are  structured  to  ensure  that  a 
team  is  integrating  the  appropriate  MT  into  one  of  the  three  CCOs  previously  discussed. 
Following  these  steps  not  only  ensures  the  proper  used  of  a  priori  and  a  posteriori  metrics, 
but  also  that  the  three  CCOs  are  continually  achieved  and  due  diligence  is  being  rigorously 
applied  when  necessary. 
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Figure  4.  The  6-3-5  Method’s  Due  Diligence  Steps 

These  five  steps  also  establish  and  maintain  an  environment  that  catalyzes  the 
workforce  to  collaborate  and  discover  innovative  solutions  to  uncovered  challenges  that 
would  have  impacted/compromised  the  reliability  and  quality  of  the  products/services  being 
delivered. 


DDS-1 :  Provide  Direction — Step  1  focuses  on  having  management  use  a 
checklist  to  make  sure  that  adequate  direction  is  provided  for  any  follow-on 
assessments  and  analysis.  It  also  includes  using  lessons  learned,  either 
statistically  or  assessed  heuristically,  to  identify  weaknesses  in  the  direction. 
Step  1  is  crucial  to  include  if  CCO-1  is  to  be  achieved.  Metadata  tags  will  use 
MT-1  for  statistic-based  gap  analysis  and  MT-3  to  validate  problem-solving 
solution. 

DDS-2:  Readiness  of  Workforce  to  Succeed — Step  2  is  the  due  diligence 
process  to  ensure,  within  reason,  that  the  workforce  is  (1)  set  up  to  succeed, 
(2)  handle  the  unexpected,  and  (3)  able  to  support  each  other  when  faced 
with  severe  challenges.  Once  Step  1  involving  direction  is  complete,  where 
strengths  and  weaknesses  of  the  direction  provided  are  known  based  on 
lessons  learned,  the  implementers  need  to  proceed  with  due  diligence  is 
resolving  issues.  A  detailed  due  diligence  process  is  described  in  Case  Study 
2  focused  on  workforce  innovation  and  alignment.  Even  if  no  lessons  learned 
issues  arise  from  Step  1 ,  to  ensure  “best  chance  to  succeed,”  the  6-3-5 
Method  supports  workforce  self-assessment  of  assigned  tasks  in  terms  of 
their  experience,  skills,  and  other  factors  associated  with  reliable,  quality 
results.  (Specific  factors  and  algorithm  structure  for  this  self-assessment 
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process  is  outside  the  bounds  of  this  paper.)  Step  2  is  crucial  to  include  if 
CCO-2  is  to  be  adequately  achieved.  Use  of  metric  visibility  regarding  due 
diligence  rigor  will  be  provided  within  the  case  studies.  Metadata  tags  will  use 
MT-2  for  heuristic-based  gap  analysis  and  MT-3  to  validate  problem-solving 
solution. 

DDS-3:  Measure  Results — Once  tasks  are  completed,  typical  performance 
results  are  measured,  including  Full  Time  Equivalent  (FTE)  hours  and 
schedule  (calendar  duration  of  tasking)  comparing  what  was  estimated  to 
what  actually  occurred.  Step  3  is  crucial  to  include  if  CCO-2  is  to  be  achieved. 

DDS-4:  Learn  from  Results — This  step  is  another  form  of  due  diligence 
focused  on  learning  from  results.  The  6-3-5  Method  requires  that  due 
diligence  include  deliberate  learning.  The  goal  to  Step  4  is  to  shift  the  focus  of 
the  learning  from  cost  overruns  and  schedule  delays  to  internal  factors  (i.e., 
ways  to  better  provide  direction,  more  accurate  workforce  alignment 
assessments  that  solve  cost  variance  issues,  including  performance).  Step  4 
is  crucial  to  include  if  CCO-3  is  to  be  achieved.  Metadata  tags  will  use  MT-4 
for  data  understanding,  MT-5  to  validate  problem-solving  solution,  and  MT-6 
to  capture  learning. 

DDS-5:  Apply  Consequences — In  the  real  world  or  a  classroom, 
consequences  are  part  of  the  educational  process.  Step  5’s  focus  is  to  use 
what  is  learned  in  Step  4  to  determine  rewards  for  success  and  next  step 
learning  for  those  failed  expectations.  Step  5  is  crucial  to  include  if  CCO-3  is 
to  be  achieved.  Metadata  tags  may  use  MT-5  to  validate  problem-solving 
solution  and  MT-6  to  capture  learning. 

Given  the  statistical  processing  and  heuristics  involved  within  the  6-3-5  Method,  it 
was  necessary  to  develop  a  metadata-driven  “Workforce  Alignment  to  Business 
Expectations”  (WA2BE)  Algorithm  incorporating  all  6  MTs,  3  CCOs,  and  5  DDSs.  The 
algorithm  has  been  applied  to  waterfall  project  management  and  Agile  software 
development  styles.  Independent  of  the  version,  the  algorithm  consists  of  two  parts.  The  first 
part  (Figure  5)  uses  a  priori  metrics  to  identify  gaps  in  a  proposed  cost.  The  first  part  is 
before  task  execution  but  can  be  run  up  until  the  time  tasks  are  assigned  for  the  resource 
talent.  The  second  part  (Figure  6)  of  the  algorithm  uses  a  posteriori  metrics  that  are  applied 
after  the  resource  talent  is  performing  the  assigned  task.  The  algorithm  uses  performance 
data  during  and  after  implementation. 

Uniquely,  the  WA2BE  Algorithm  knows  how  to  translate  “similar”  historical  data  and 
make  it  relevant  to  a  current  project  or  operations.  “Similar”  is  based  on  various  types  of 
complexity  parameters  of  skill  set  and  workload.  The  following  three  case  studies 
demonstrate  how  various  aspects  of  the  6-3-5  Method  incorporating  the  metadata-based 
algorithm  is  applied  to  the  acquisition  and  management  process,  and  the  results  achieved 
from  the  application. 
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Figure  5.  WA2BE  Algorithm  Process  Flow  Before  Implementation 


Figure  6.  WA2BE  Algorithm  Process  Flow  During/After  Implementation 
PEO  Carriers  Case  Study 

In  this  NAVSEA-aligned  PEO  case  study,  statistically-based  lessons  learned 
metadata  is  used  to  support  CCO-3.  In  this  case  study,  previous  RCOH  data  allowed  the 
WA2BE  algorithm  to  compare  identical  SWLIN  types  resulting  in  gap  analysis  of  contract 
performance  reliability.  Note  that  in  cases  where  the  same  type  of  data  is  absent  for  past 
performance  analysis,  then  the  WA2BE  algorithm  can  translate  and  use  “similar”  past 
performance  data  to  identify  issues  prior  to  contract  award  with  statistical  confidence. 

The  statistical  analysis,  implemented  by  the  WA2BE  algorithm,  analyzed  various 
permutations  within  the  historical  performance  data  to  determine  “best”  estimation 
characteristics.  A  Lambda  Level  6  (Figure  1)  was  selected  for  the  algorithm’s  analysis.  An 
important  note:  Once  the  statistically-based  lessons  learned  are  applied  to  identify  gaps,  the 
five  steps  involving  due  diligence  discussed  previously  can  then  be  applied  to  eliminate  or 
mitigate  cost  control  issues. 

In  the  April  2015  online  edition  of  the  Navy  League’s  Seapower  magazine,  The 
Honorable  Frank  Kendall,  Linder  Secretary  of  Defense  for  Acquisition,  Technology,  and 
Logistics,  stated,  “We  will  not  start  programs  we  cannot  afford”  (Kendall,  2015).  The  U.S. 
Government  Accountability  Office  (GAO)  recommends  cost  control  methods  described  in  its 
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Defense  Major  Automated  Information  Systems:  Cost  and  Schedule  Commitments  Need  to 
Be  Established  Earlier  (2015)  report.  That  study  emphasizes  the  need  to  have  programs 
establish  their  initial  acquisition  program  baselines  (APBs)  involving  cost  and  schedule 
within  two  years  of  officially  reporting  that  work  has  commenced.  Industry’s  response  to 
comply  with  this  study  is  to  create  smaller,  more  manageable  programs  or  program 
segments.  Unfortunately,  as  this  white  paper  describes  through  its  case  studies,  even 
establishing  an  APB  within  two  years  to  comply  with  these  findings  is  still  not  an  adequate 
cost  control  approach.  This  becomes  obvious  when  coherent  metadata  tagging  is  created  as 
a  basis  for  comparison  between  past  overhauls  and  a  planned  or  in-process  overhaul,  as  is 
described  below. 

To  understand  why  a  two-year  APB  approach  will  not  fully  satisfy  Under  Secretary 
Kendall’s  stated  need,  this  case  study  examined  a  Nimitz-class  aircraft  carrier  class 
refueling  and  complex  overhaul  (RCOH)  of  USS  Carl  Vinson  (CVN-70).  The  RCOH  occurred 
in  2005  as  reported  in  the  July  14,  2009,  issue  of  Defense  Industry  Daily  (DID)  (CVN  70, 
2009).  That  article  stated,  “In  November  2005,  [at  that  time]  Northrop  Grumman  Newport 
News  [Shipbuilding  (NNS) — now  Huntington  Ingalls  Industries]  in  Newport  News,  VA  was 
awarded  a  $1 .94  billion  cost-plus-incentive-fee  [CPIF]  contract  for  accomplishment  of  the  FY 
2006  mid-life  refueling  and  complex  overhaul  (RCOH)  of  the  Nimitz  Class  aircraft  carrier 
USS  Carl  Vinson  [CVN  70].”  As  that  article  further  states,  “NAVSEA’s  official  cost  figure  for 
the  CVN  70’s  entire  RCOH  is  $3.1  billion.  As  of  April  2007,  they  [NAVSEA/PEO  Carriers] 
told  DID  that  the  program  was  on  budget.”  Thus,  within  less  than  a  two-year  period 
(November  2005  to  April  2007),  the  cost  had  increased  by  over  $1  billion. 

Timeline  for  the  USS  Carl  Vinson  RCOH: 

•  4th  Quarter  CY2005 — “Workforce  Alignment  to  Business  Expectations” 

(WA2BE)  Algorithm  was  applied  and  used  a  grading  system  of  an  “A”  through 
“F”  for  each  of  the  CVN-70  Carl  Vinson’s  RCOH  Ships  Work  List  Item 
Number  (SWLIN).  Grades  of  C,  D,  E,  and  F  indicated  that  cost  overruns 
would  potentially  exceed  contract  RCOH  goals.  Using  a  stoplight  dashboard, 
C  is  represented  as  “yellow,”  where  D,  E,  and  F  are  displayed  with  “red.”  The 
grades  are  color-coded  in  Table  1.  The  algorithm  had  an  overall  statistical 
confidence  of  99%  that  the  RCOH’s  costs  would  exceed  its  contract  goals 
(coded  “red”): 

o  Sample  Source:  The  statistical  analysis  was  based  on  684  SWLIN 
forecasted  grades  using  CVN-68  and  CVN-69  RCOH  historical  data 
that  was  provided  by  PEO  Carriers.  There  were  seven  SWLINs  that 
had  no  values  available  for  analysis. 

o  Metadata  Summary  (Contributors  to  cost  overruns — Cs,  Ds,  Es,  and 
Fs  are  tags  to  indicate  the  degree  in  which  actuals  will  potentially 
exceed  contractual  agreement — the  lower  the  grade,  the  higher  the 
potential):  Table  1  is  a  simple  example  of  metadata  analysis  using 
previous  overhauls  per  SWLIN  categorization  as  linked  to  shops 
assigned,  trades  assigned,  and  related  management/operations. 
These  types  of  analytical  summaries  based  on  a  priori  metrics  allow 
for  better  bid  negotiations. 
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Table  1.  CVN-70  RCOH  Metadata  Summary  Analysis  of  Bid  From  Newport  News 

Shipbuilding 


•  4nd  Quarter  CY2005 — Newport  News  Shipbuilding  (NNS)  Awarded  $1 .94 
Billion  CPIF  Contract 

•  2nd  Quarter  CY2007 — NAVSEA/PEO  Carriers’  Official  [Actual]  Cost  is  $3.18 
Billion  [Reported  by  DID],  obviously  exceeding  the  original  contract  goals. 

Figure  7  shows  three  figures,  where  each  figure  represents  a  row  in  Table  1.  The 
three  figures  describe  probability  density  functions  determined  using  the  APRI  analytics  and 
maximum  likelihood  criteria  to  statistically  reduce  false  positive  results.  The  RCOH  bid 
contractually  needed  to  be  within  +/- 10%.  684  SWLINs  had  a  statistical  confidence  of  80% 
or  greater  that  they  had  either  a  marginal  or  poor  estimate  in  terms  of  meeting  the 
contractual  10%  threshold.  This  analysis  involved  shops,  trades,  and  management.  This  is 
why  the  WA2BE  Algorithm  determined  with  a  statistical  confidence  interval  of  99%  that  the 
costs  would  exceed  the  contractual  threshold.  Use  of  the  WA2BE  Algorithm  demonstrated 
that  this  RCOH  bid  should  not  be  accepted  without  all  684  SWLINs  reviewed  for 
improvement.  Starting  with  the  “F”  graded  SWLINs,  the  review  needs  to  use  the  algorithm’s 
analytics  to  determine  if  any  changes  were  effective  (creating  an  “A”  or  “B”  grade)  in  having 
the  U.S.  Navy  avoid  getting  “stuck  with  the  bill” — again! 


Percent  where  Statistical  Confidence  Analysis  was 
Greater  than  80%  for  RCOH  Managenaent/Ops  Bids 


Percent  Over  Bid 


Percent  where  Statistical  Confidence  Analysis 
was  Greater  than  80%  for  RCOH  Shop  Bids 


Percent  where  Statistical  Confidence  Analysis 
was  Greater  than  for  rcoh  Trade  Bids 

5?*  4i%  i% 


Percent  Over  Bid 


Figure  7.  Probability  Density  Function  View  of  Metadata  in  Table  1 

Figure  8  provides  an  example  of  how  the  results  from  using  five  due  diligence  steps 
within  the  methodology  are  statistically  validated.  The  key  is  to  be  able  to  graphically  view 
the  results  of  the  due  diligence  efforts  with  stoplight  displays  before  proceeding  with  bid 
acceptance  or  task  management.  PEO  Carriers  did  not  follow  the  five  due  diligence  steps. 
Therefore,  it  is  difficult  to  determine  whether  they  accepted  the  bid  under  Figure  7  conditions 
or  those  of  Figure  8. 
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Figure  8.  Target  Goal  of  a  Bid  Statistical  Analysis 

To  emphasize  when  using  the  6-3-5  Method,  Figure  8  provides  an  example  of  what 
the  statistical  analysis  of  any  RCOH  bid  must  be  before  the  contract  is  accepted.  The 
following  case  study  describes  how  the  6-3-5  Method  supports  the  achievement  of  this  goal 
when  given  issues  represented  in  Figure  7. 

NATO  Program  Case  Study 

The  6-3-5  Method’s  WA2BE  Algorithm  applied  heuristic  processing  to  a  NATO- 
sponsored  program  developing  Air  Traffic  Control  Systems  (ATCSs)  for  various  nations, 
including  Belgium.  The  prime  contractor  developing  the  systems  was  a  Hughes  Aircraft 
spinoff  company.  The  ATCSs  being  developed  required  the  use  of  touchscreens  (a  new 
technology  at  the  time)  and  had  the  complexity  of  integrating  new  hardware  development 
with  complex  software  coding.  As  described  below,  the  prime  contractor  encountered  many 
difficulties  creating  an  accurate  baseline  for  the  Belgian-variant  ATCS  acquisition — until  the 
WA2BE  algorithm  was  applied.  These  problems  were  addressed  using  a  formal  workforce 
collaboration  problem-solving  approach  supported  within  the  6-3-5  Method  as  described  in 
Figure  9. 

The  6-3-5  Method’s  collaboration  problem-solving  approach  was  successfully  used 
by  the  prime’s  management  team  to  reduce  cost  overruns  dramatically  from  35%  to  3%. 
When  following  the  6-3-5  Method,  first  the  implementation  of  the  metrics  allowed  for  visibility 
into  issues.  The  key  problem  discovered  was  a  requirement  document  that  did  not  provide 
enough  detail  for  the  workforce  to  assess  their  alignment.  That  means  that  there  was  no 
alignment,  which  caused  more  than  80%  of  the  task  to  be  in  the  red,  as  described  in  Figure 
10. 


After  the  requirements  document  was  written  to  allow  the  workforce  being  assigned 
tasks  to  self-assess,  the  following  corrective  actions  were  taken  based  on  the  results  from 
the  workforce  applying  the  6-3-5  Method’s  collaboration  problem-solving  approach  (Figure 
9)  to  improve  cost  control  (i.e.,  workforce  alignment): 

•  Increase  the  team’s  confidence,  knowledge,  or  training  about  performing  the 
Tasks  per  management’s  productivity  and  efficiency  expectations  (a)  informal 
training,  (b)  formal  training,  (c)  completing  similar  Tasks. 

•  Modify  the  resources  (e.g.,  internal  resources,  external  resources,  time, 
labor-hours)  or  working  environment  defined  for  use  with  the  assigned  Tasks. 
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•  Modify  the  way  the  team  is  accomplishing  their  Tasks  by  using  a  different 
approach. 

•  Modify  the  support  group/other  team  members  to  assist  each  other  in  doing 
assigned  Tasks  more  reliably. 

•  Increase  incentives  for  the  team  to  perform  the  complexities  of  the  Tasks. 

It  is  important  to  note  that  had  the  Nunn-McCurdy  Act  applied  to  this  NATO  program, 
it  would  have  been  well  within  the  15%  requirement,  and  even  when  the  Amendment  was  at 
10%  (Nunn-McCurdy  Act,  1983).  The  DoN  would  not  have  been  stuck  with  another 
unexpected  bill,  as  described  in  Case  Study  1.  Instead,  this  program  used  all  six  MTs  as 
they  continually  cycled  through  the  five  DDSs,  achieved  all  three  CCOs,  enabling  NATO 
prime  contractor  management  to  realize  greater  cost  control  by  meeting  budget,  schedule, 
and  quality  expectations  more  reliably.  The  NATO  program  was  being  due  diligently 
supported  (see  Figure  10). 
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Figure  9.  Problem  Solving  Collaboration  Approach 
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Figure  10.  Heuristic  Assessment  Analysis  Statistically  Mapped  to  APRI  Table 
Before  and  After  Problem  Solving  Collaboration  Approach  Validating 
Improved  Cost  Control  Resulting  in  Cost  Overrun  Reduction 

From  35%  to  3% 

The  WA2BE  Algorithm’s  implementation  of  the  five-step  method  supports  a  rigorous 
due  diligence  approach  involving  team  problem  solving  through  collaboration  that  promotes 
team  innovation.  Figure  9  provides  how  collaboration  can  lead  a  team  to  innovative 
solutions  and  lessons  learned.  Using  this  process  flow  results  in  one  of  three  results: 

1 .  A  solution  that  does  not  require  innovation 

2.  A  solution  that  does  require  innovation 

3.  A  next  step  to  discover  a  solution 

If  the  problem  is  not  solved  in  time,  the  algorithm  supports  the  team  with  lessons 
learned  documentation  of  the  steps  used  and  the  related  outcomes  to  discovering  the 
solution.  In  the  case  where  the  issue  happens  again,  the  workforce  can  access 
documentation  to  use  as  a  running  start  at  finding  the  solution  within  the  needed  time  frame. 
If  a  solution  is  found,  the  lessons  learned  provides  immediate  assistance  to  support  the 
workforce  in  knowing  how  to  recover  from  any  impact  that  would  increase  cost  variance. 
Lesson  learned  documentation  is  in  the  form  of  what  worked  and  what  did  not  work. 

The  hypothesis  is  offered:  Can  any  team  be  due  diligent  about  reducing  impact  of 
unknowns  to  a  project  without  considering  innovative  options/solutions  to  the  issues?  Would 
this  be  considered  a  significant  workforce  collaboration  goal?  The  algorithm’s 
implementation  of  the  6-3-5  Method  is  based  on  the  assumption  that  workforce 
collaboration,  and  when  needed,  workforce  innovation,  is  essential  to  success  when 
attempting  to  achieve  rigorous  due  diligence. 

Highlighting  a  difference  in  approach  as  compared  to  the  CVN-70  RCOH  case  study, 
the  prime’s  executive  leadership  for  the  Belgian  ATCS  chose  to  make  the  time  investment  to 
perform  the  necessary  due  diligence — despite  already  being  behind  schedule — to  address 
the  “red”  stoplight  issues.  First,  the  algorithm  was  used  to  achieve  CCO-1  (Setting  and 
managing  customer/business  expectations  for  the  program).  Next,  the  algorithm  was  used 
to  support  CCO-2  (Reliably  achieving  those  expectations).  The  algorithm  allowed  the  team 
to  assess  their  reliability  in  meeting  expectations,  with  objective,  metric  precision.  Once 
assessed,  the  algorithm  identified  gaps  in  workforce  alignment,  prioritizing  these  gaps  in 
terms  of  criticality  (i.e.,  greatest  budget,  schedule  and  quality  impact  to  the  program), 
allowing  the  team  to  focus  on  worst  case  scenarios.  This  was  all  done  through  its  tagging 
process.  The  algorithm  facilitated  constructive  discussions  by  having  tags  that  provided 
simpler  descriptions  of  core  issues  that  allowed  the  performing  team  to  find  innovative 
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solutions,  when  needed.  This  gave  the  performing  team  an  advantage  in  meeting 
expectations  more  reliably,  producing  wiser,  more  efficient  and  when  needed,  innovative 
solutions. 

The  WA2BE  algorithm’s  connectivity  relationships  and  related  metadata  tagging 
made  it  easier  for  the  team  to  innovatively  solve  issues — turning  an  average  group  into  a 
high  performance  team. 

For  example,  the  NATO-sponsored  ATCS  development  program  required  each 
system  being  tailored  for  various  countries  be  designed  with  touchscreen  user  interfaces. 
Unfortunately  for  the  Belgian  ATCS  program,  the  prime’s  software  development  team  didn’t 
have  access  to  touchscreen  user  interfaces  at  their  workstations,  nor  was  funding  available 
to  upgrade  their  workstations.  For  the  Belgian  team,  the  touchscreens  were  only  available 
on  the  Test  and  Evaluation  (T&E)  Test  Bed,  which  was  time  shared  by  multiple  NATO- 
sponsored  programs  for  formal  integration  testing.  The  algorithm  identified  “lack  of  funding 
to  buy  touchscreen  user  interfaces”  as  a  symptom  of  the  Belgian  team’s  inability  to  meet 
cost  and  schedule.  Additionally,  like  all  symptoms,  it  did  not  present  itself  as  a  solvable 
problem  for  the  team  to  address.  In  fact,  it  caused  them  to  feel  victimized  given  other 
circumstances  associated  with  the  prime’s  executive  leadership.  This  is  a  very  typical 
workforce  response  when  they  feel  that  they  are  set  up  to  fail  by  management. 

The  complete  symptom  presented  itself  as  follows:  For  the  Belgian  ATCS  software 
team,  workflow  involved  code  development  at  their  workstations,  followed  by  software 
installation  onto  the  T&E  Test  Bed.  Once  installed,  they  ran  their  code  using  the  timeshared 
Test  Bed’s  touchscreens  to  find  “bugs.”  Once  found,  they  needed  to  end  their  Test  Bed 
session  to  return  to  their  workstations  to  “hopefully”  fix  the  code.  Once  potentially  fixed,  they 
had  to  reinstall  their  updated  version  onto  the  Test  Bed,  again  timesharing  with  other  ATCS 
programs,  and  check  their  code  using  the  T&E  Test  Bed’s  touchscreens.  Then  the  process 
repeated  to  resolve  any  new  bugs  found.  Until  all  bugs  were  fixed,  formal  integration  tests 
could  not  be  performed.  Because  of  the  timesharing  challenges,  this  caused  the  Belgian 
team  to  incur  uncontrollable  cost  overruns  and  schedule  strips. 

The  algorithm  forecasted  the  number  of  days  these  symptoms  would  push  the 
schedule,  thus  causing  a  failure  in  meeting  requirements.  The  algorithm’s  metadata  process 
converted  these  symptoms  into  workforce  alignment  gaps  that  identified  core  issues  based 
on  one  of  the  five  alignment  factors  in  meeting  business  expectations,  breaking  down  the 
problem  into  addressable  and  solvable  “chunks.” 

These  problem  “chunks,”  customized  to  the  team’s  specific  workforce  alignment 
needs,  were  then  filtered  into  a  solvable  format  for  the  team  to  more  easily  and  efficiently 
apply  their  collective  wisdom  and  innovation.  This  translation  into  smaller,  solvable  issues 
allowed  the  team  to  creatively  improve  workforce  alignment.  In  other  words,  the  algorithm’s 
use  of  metadata  guided  the  team  to  innovate  a  solution  based  on  their  alignment  needs  in 
meeting  schedule,  without  affecting  cost  or  quality. 

Whether  applying  the  WA2BE  algorithm  to  an  S-Curve  analysis,  Earned  Value 
Management  (EVM),  etc.,  the  new  baseline/target  was  assessed  with  the  quality  of  data 
having  mostly  “As”  and  “Bs,”  with  a  few  “Cs”  that  were  being  mitigated,  as  opposed  to  its 
previous  lower  grades  (Earned  Value  Management  Systems,  2007).  The  “garbage  in, 
garbage  out”  syndrome  was  eliminated.  The  new  baseline  was  significantly  different  in 
another  way;  it  conformed  to  the  provided  funding,  once  thought  insufficient.  The  data 
supporting  any  type  of  analysis,  S-Curve,  EVM,  etc.,  now  consists  mostly  of  all  “green” 
lights — indicating,  “quality  in,  quality  out.”  As  indicated  in  this  case  study,  the  Belgian  team 
faced  and  innovatively  conquered  a  common  issue  in  today’s  development  environment, 
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“lack  of  funding.”  During  the  life  of  the  program,  they  continued  to  solve  issues  using  the 
Just-in-Time  Problem  Solving  Collaboration  Approach  instead  of  feeling  victimized. 

The  transformation  of  the  Belgian  team  in  its  ability  to  meet  expectations  was  so 
dramatic  that  the  prime  contractor  began  applying  the  WA2BE  Algorithm  to  its  other  NATO- 
sponsored  programs. 

Naval  Warfare  Center  Case  Study 

Performing  under  Navy  fixed  price  support  service  contracts,  the  WA2BE  algorithm 
was  applied  within  various  branches  of  two  NAVSEA  Warfare  Centers  of  Excellence 
(COEs).  For  both  centers,  since  historical  data  was  not  available  to  apply  the  statistical 
analyses,  assessments  were  made  via  MT-2  and  heuristically  analyzed  using  the  WA2BE 
Algorithm.  MT-2  supported  DoN  civil  service  employees  and  related  contractors  to  create 
the  data  that  was  translated  into  metadata  and  mapped  to  an  APRI  reference,  as  was  done 
in  Case  Study  2.  Case  Study  3  will  compare  the  use  of  WA2BE  Algorithm  by  branches  from 
two  different  COEs. 

For  both  warfare  centers,  the  first  challenge  was  in  the  achievement  of  CCO-1 
(Setting  and  managing  expectations).  At  one  center’s  branch,  a  more  effective  WBS  was 
created,  with  a  nomenclature  that  supported  greater  understanding  of  the  relationships 
within  the  branch’s  organization.  At  the  other  center’s  branch,  where  deliverables  were 
defined  in  Task  Planning  Sheets  (TPSs),  the  focus  was  on  documenting  workflow  and 
defining  FTE  hours  consumed.  For  both  branches,  once  metrically  determined  as  having 
CCO-1  complete  with  the  appropriate  metadata  analysis,  the  algorithm  supported 
management  to  achieve  CCO-2  (Reliably  achieving  expectations). 

Without  prior  history,  the  algorithm  generated  forms  based  on  the  metadata  for  both 
branches  to  self-assess  workforce  alignment  to  deliver  their  assigned  fleet  work  products 
with  reliability  and  quality.  In  CCO-2,  the  algorithm  used  the  completed  forms  to  identify 
gaps  in  workforce  alignment  and  then  used  the  Just-in-Time  Problem-Solving  Methodology 
to  guide  the  groups  in  addressing  their  respective  issues  without  compromising  time,  money 
or  quality.  One  center’s  branch  manager  went  through  detailed  preparation  to  use  the  Just¬ 
in-Time  Problem-Solving  Methodology  based  on  the  alignment  metrics.  Another  center’s 
branch  manager  chose  not  to  go  through  the  training.  The  results  were  profoundly  different 
in  a  manager’s  ability  to  grasp  innovative  due  diligence  performed  by  his  team  whenever 
dealing  with  challenges. 

It  is  important  to  note  that  when  a  branch  manager  chose  not  to  receive  training,  the 
solutions  produced  by  his  team  to  improve  quality  of  deliverables  were  suppressed. 
Presumably,  the  branch  manager  was  concerned  that  his  senior  leadership  would  view  this 
data  as  an  inability  to  manage  his  group  effectively.  This  is  another  example  of  SECNAV’s 
comments  and  concerns  regarding  a  long  standing  “zero-defect”  culture. 

Significantly,  when  the  team’s  solutions  to  deal  with  “red”  stoplight  issues  were  not 
supported  by  management,  the  identified  issues  manifested  within  six  months  to  a  year. 

This  resulted  in  formal  negative  feedback  from  the  branch’s  Navy  customer.  Again,  this 
emphasizes  two  points:  (1)  even  without  historical  data,  the  algorithm  assessment  using  the 
metadata  approach  was  shown  to  be  able  to  forecast  issues  accurately  six  months  in 
advance,  and  (2)  when  “red”  stoplight  issues  are  identified  and  solutions  provided  within  the 
metadata  structure,  they  need  to  be  addressed  and  implemented  before  impact. 

Timeline  for  the  branch  that  did  not  support  team’s  solutions  to  “Red”  stoplight 

issues: 
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•  CY2007 — “Workforce  Alignment  to  Business  Expectations”  (WA2BE) 
Algorithm  used  a  grading  system  of  an  “A”  through  “F”  for  each  of  the 
deliverables  called  out  by  the  TPSs.  Grades  C,  D,  E,  and  F  indicated 
potential  quality  issues.  Using  a  stoplight  dashboard,  C  is  represented  as 
“yellow,”  where  D,  E,  and  F  are  displayed  as  “red.”  The  grades  are  color- 
coded  in  Table  2.  Using  the  metadata  structure,  the  algorithm  produced 
assessments  that  were  answered  by  branch  team  members  and  analyzed 
based  on  APRI  and  metadata  relationships.  (The  just-in-time  innovation 
solutions  to  improve  the  quality  of  the  work  generated  by  the  branch 
workforce  assigned  to  the  deliverables  were  discarded.) 

o  Sample  Source:  The  analysis  is  based  on  181  deliverables  and  their 
related  assessments.  There  were  four  deliverables  that  had  no 
assessments  available.  Appropriation  source  were  Operations  and 
Maintenance,  Navy  (O&MN) — funding  60  deliverables;  Other 
Procurement,  Navy  (OPN) — funding  40  deliverables;  Ship 
Construction  &  Conversion,  Navy  (SCN) — funding  54  deliverables; 
and  Navy  RDT&E — funding  31  deliverables  being  analyzed  by  the 
algorithm. 

o  Metadata  Summary  (Contributors  to  Quality  Issues — Cs,  Ds,  Es,  and 
Fs  are  tags  to  indicate  the  degree  in  which  actual  performance  will 
potentially  have  quality  issues — the  lower  the  grade,  the  higher  the 
potential):  Table  2  is  a  simple  example  of  metadata  analysis,  where 
each  type  of  money  connects  to  its  related  deliverables,  related 
processes,  people  assigned  within  the  process,  and  related  alignment 
metrics  per  tasks/activities  along  the  process.  This  data  is  tagged  and 
summarized  below.  These  types  of  analytical  summaries  allow  for 
better  management  of  monies  within  branches,  divisions, 
departments,  commands,  and  various  other  echelons  within  the  DoN. 


Table  2.  Metadata  Summary  Analysis  of  a  NAVSEA  CEO  Branch’s  Quality  of 

Deliverables  by  Appropriation 
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•  CY2008 — The  NAVSEAfPEO  Customer”  had  significant  complaints  involving 
quality  of  work  performed  by  that  same  specific  branch  that  used  the 
algorithm  approximately  six  months  earlier  but  discarded  the  team’s  solutions 
to  improve  the  quality  of  work. 

•  CY2014 — Executive  leadership  for  the  entire  Center  was  replaced  due  to 
Navy  customer  complaints  regarding  the  quality  of  work. 

Significantly,  when  the  team’s  solutions  to  deal  with  “red”  stoplight  issues  were  not 
supported  by  management,  the  identified  issues  manifested  within  six  months  to  a  year. 

This  resulted  in  formal  negative  feedback  from  the  branch’s  Navy  customer.  Again,  this 
emphasizes  two  points:  (1)  even  without  historical  data,  the  algorithm  assessment  using  the 
metadata  approach  was  shown  to  be  able  to  forecast  issues  accurately  six  months  in 
advance;  and  (2)  when  “red”  stoplight  issues  are  identified  and  solutions  provided  within  the 
metadata  structure,  they  need  to  be  addressed  and  implemented  before  impact. 


Quality 

Needs 


Percent  where  Statistical  Confidence 
Analysis  was  Greater  than  80%  for 
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Figure  11.  Heuristic  Assessment  Analysis  Statistically  Mapped  to  APRI  Table 
Before  and  After  Problem  Solving  Collaboration  Approach  Validating 
Proposed  Solution  to  Improving  Workforce  Quality 

As  Deming  (1993)  once  stated,  “A  bad  system  will  beat  a  good  person  every  time.” 
For  the  branch  manager  who  did  not  allow  “red”  stoplight  issues  to  be  solved  and 
implemented  by  their  teams,  was  this  a  display  of  a  “zero-defect,  never  failing”  mentality, 
and  was  it  pervasive  throughout  his  chain  of  command?  In  support  of  understanding  the 
culture  of  the  command,  there  is  another  instance  when  the  center  used  the  algorithm  to 
support  implementing  NAVSEA-wide  initiatives.  In  CY2014,  when  a  high-level  civil  service 
employee  remarked  about  his  group’s  “red”  stoplight  issues,  “Don’t  throw  me  under  the  bus 
for  my  answers.”  He  preferred  to  avoid  stating  that  “the  emperor  has  no  clothes”  and  any 
consequences  that  might  ensue. 

The  algorithm  using  its  metadata  structure  accurately  determined  that  the  workforce 
alignment  was  so  poor  that  his  implementation  team  had  a  very  low  likelihood  of  meeting  the 
defined  expectations  of  the  command  in  supporting  the  NAVSEA-wide  initiatives.  Once 
again,  in  Case  Study  3,  the  concern  was  what  executive  leadership  might  think  of  his 
answers.  When  executive  leadership  reviewed  his  assessment,  their  predominant  concern 
was  what  impression  a  visiting  NAVSEA  admiral  might  think  of  their  command’s 
organization.  This  supports  the  belief  that  a  pervasive  “zero-defect,  never  failing”  mentality 
was  being  evidenced  throughout  the  chain  of  command. 
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Figure  1 1  demonstrates  that  while  49%  supported  by  this  branch  are  of  good  quality, 
51%  had  a  statistical  confidence  of  80%  or  greater  that  the  work  quality  would  be  marginal 
or  poor.  When  civil  service  work  quality  suffers,  the  impact  can  seriously  hinder  the  U.S. 
Navy’s  readiness  to  support  its  warfighting  missions. 

Failure-focused,  “zero-defect”  cultures  must  end  if  cost  control  is  to  improve. 
Unfortunately,  ignoring  or  at  worst,  discarding  data  that  accurately  forecasts  outcomes  does 
not  avoid  issues;  it  only  delays  having  to  deal  with  them.  Such  delays  make  any  corrective 
action,  after  the  fact,  more  costly. 

The  algorithm’s  metadata  approach  to  identify  and  overcome  issues,  instead  of  being 
victimized  by  them,  directly  contrasts  against  a  non-zero  defect  mentality.  In  Case  Study  3, 
one  center’s  branch  manager  and  his  supervisory  task  leads  took  the  training  and  benefited 
from  the  algorithm’s  analytics.  Another  center’s  branch  manager  skipped  the  training  and 
rejected  the  benefits  of  the  just-in-time  innovation  solutions  offered  by  his  team.  Can  an 
assumption  be  made  that  if  the  branch  manager  had  chosen  to  attend  the  training  instead  of 
skip  it,  would  he  have  allowed  his  team  to  follow  through  with  their  just-in-time  innovation 
solutions? 

This  case  study  emphasizes  how  the  WA2BE  Algorithm  fills  a  gap  as  a  needed 
solution  for  use  by  various  U.S.  Navy  shore  commands  as  part  of  their  cost  control 
approach,  supporting  just-in-time  innovation.  All  commands  and  programs  within  the  DoN 
need  stabilized  budgets,  where  innovation  is  used  to  maintain  budget  adherence  reliably  to 
meet  expectations.  This  algorithm  supports  this  type  of  cultural  change. 

Conclusions 

The  data  collected  at  the  SWLIN/TPS  deliverable  levels,  once  metadata  is  tagged 
and  processed  through  the  algorithm,  becomes  very  revealing.  The  three  case  studies 
reveal  crucial  gaps  in  the  DoN’s  current  ability  to  achieve  an  effective  cost  control  solution. 
Two  of  these  case  studies  were  performed  under  NAVSEA  contracts  and  the  third  in  support 
of  a  NATO  program.  The  case  studies  demonstrate  how  the  metadata-based  algorithm 
implementing  the  6-3-5  Method  gave  leadership  greater  proactive  control  over  internal  and 
external  factors  that  affected  cost  variance.  Specifically,  the  algorithm  identifies  and  collects 
workforce  alignment  data,  making  relevant  links  between  data  points,  providing  heuristics 
(an  approach  to  achieve  Artificial  Intelligence)  to  create  tags  that  summarize  the  value  of 
those  links,  and  processes  analytically  the  metadata  tags  to  provide  statistical  confidence 
and  other  higher-level  metadata  related  to  cost  control. 

Through  application  of  the  algorithm’s  automated  analysis,  as  shown  via  tables  of 
cost  control  data  collected  from  these  studies,  leadership  is  able  to  make  more  informed 
decisions  and  an  aligned  workforce  is  encouraged  and  enabled  by  management  to 
contribute  through  innovation  and  by  lending  its  fully  invested  and  supportive  voice  to 
recommend  process  improvements.  These  solutions  offer  management  improved  decision¬ 
making  capability  to  better  tailor  the  program  for  success. 

As  per  H.  James  Harrington,  PhD,  president  and  chairman  of  the  American  Society 
for  Quality  (ASQ)  and  president  and  chairman  of  the  International  Academy  for  Quality 
(IAQ),  “Measurement  is  the  first  step  that  leads  to  control  and  eventually  to  improvement.  If 
you  can’t  measure  something,  you  can’t  understand  it.  If  you  can’t  understand  it,  you  can’t 
control  it.  If  you  can’t  control  it,  you  can’t  improve  it”  (Harrington,  1987,  p.  43).  Through  the 
6-3-5  Method’s  metadata  analysis  via  the  WA2BE  Algorithm  based  on  either  statistic  or 
heuristic  assessment  measurements  for  each  program  and  aggregating  in  the  roll-up  from 
lower  to  higher  Echelon  command  levels,  and  ultimately  Budget  Submitting  Offices  and 
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OPNAV  Resource  Sponsors.  These  6-3-5  Method-based  analyses  can  feed  the  related  line 
items  in  the  DoN’s  POM  submission  to  DoD  in  support  of  the  federal  government’s  Planning, 
Programming,  Budgeting  and  Execution  (PPBE)  process  (Office  of  the  Under  Secretary  of 
Defense  for  Acquisition,  Technology,  &  Logistics  [OUSD(AT&L)],  2013). 

Table  3  provides  an  example  of  how  a  stoplight  approach,  based  on  the  WA2BE 
Algorithm’s  analytics,  can  support  the  DoN’s  POM  submission  process.  The  table  represents 
a  graphical  view  of  an  original  estimate  submission  before  and  after  applying  the  algorithm’s 
statistical  confidence  analytics.  The  WA2BE  Algorithm  is  able  to  use  either  same  or  similar 
past  performance  data  related  to  each  DoN  POM  submission  line  item  to  determine  the 
statistical  accuracy  of  each  item’s  budget  estimate.  The  statistical  accuracy  of  the  analysis  is 
represented  by  graphical  stoplights,  where  each  stoplight  represents  confidence  of  80%  or 
greater  with  a  margin  of  error,  defined  by  the  Lambda  value  within  the  APRI  table.  In  an 
environment  where  budgets  need  to  accomplish  more  with  less  funding,  accurate  estimation 
using  a  priori  metadata  is  a  vital  necessity/capability  for  all  DoD  activities.  For  areas  in  the 
“red,”  the  6-3-5  Method  provides  a  proven  problem-solving  collaboration  approach,  as 
described  in  the  previous  case  studies  and  figures.  The  collaboration  approach  uses  a  priori 
metrics  and  either  heuristic  assessment  or  statistical  confidence  to  validate  the  reliability  that 
the  proposed  solution  will  cause  “red”  DoN  POM  submission  line  items  to  turn  “green,” 
assuring  accuracy  of  estimated  submission. 

Admiral  James  A.  Winnefeld  Jr.,  Vice  Chairman  of  the  Joint  Chiefs  of  Staff,  during 
his  address  to  the  Navy  League  SAS  2015  banquet,  stated  that  budget  cuts  are  causing 
significant  uncertainty  for  our  future  warfighting  capability.  More  money  is  always  a  desired 
state;  yet  even  with  more  money,  the  reliable  control  of  spending  needs  to  be  addressed.  To 
summarize  many  sentiments  discussed  by  speakers  at  SAS  2015,  including  Admiral 
Winnefeld,  there  is  a  level  of  uncertainty  with  regard  to  a  stable  budget  that  has  now 
become  a  DoD  legacy  “posing  serious  future  problems.”  In  Case  Study  1,  the  Nunn- 
McCurdy  Act  was  discussed  (Nunn-McCurdy  Act,  1983).  The  current  Act  requires  that 
Congress  be  informed  of  any  major  Acquisition  Category  (ACAT)  program  running  over 
15%.  Just  a  decade  earlier,  the  threshold  number  was  10%.  These  percentages  represent 
uncertainty  in  any  Program  of  Record’s  financial  outcome.  The  goal  should  be  to  have  less 
uncertainty,  for  example,  a  5%  threshold — not  more  uncertainty — with  regard  to  the 
mandate  of  the  Nunn-McCurdy  Act. 
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Table  3.  An  Example  of  How  6-3-5  Method  Supports  DoN  POM  Submission  in 
Knowing  Reliability  of  Submission/Risks  per  Line  Items 


Because  of  its  proven  accuracy  to  use  a  priori  metrics  to  identify  and  resolve  issue 
well  in  advance,  the  WA2BE  Algorithm  is  recommended  as  an  effective  means  to  minimize 
DoN  budget  uncertainties.  Case  Study  2,  regarding  the  reduction  of  cost  overruns  from  35% 
down  to  3%,  ensures  that  when  an  APB  is  set,  it  is  also  stable  with  a  workforce  that  is 
aligned  with  a  high  statistical  confidence  of  meeting  expectations.  Case  Study  2  also 
demonstrates  the  ability  for  the  DoN  to  establish  and  maintain  a  culture  of  innovation  and 
learning  using  the  WA2BE  algorithm  throughout  its  acquisition  and  in-service 
engineering/life  cycle  support  programs,  as  well  as  its  centers  of  excellence. 

The  6-3-5  Method  deployed  using  the  metadata-based  WA2BE  Algorithm  ensures 
that  the  workforce  remains  optimally  aligned  to  meet  fleet  and  force  capability  expectations 
reliably.  As  described  in  Case  Study  3,  this  ability  applies  to  the  federal  civil  service 
workforce  throughout  all  Navy  shore  commands  and  their  industry  service  support  partners, 
as  well  as  major  system  integrators  delivering  products.  It  ensures  that  just-in-time 
innovation  becomes  the  new  cultural  norm.  Just-in-time  innovation,  as  established  and 
visible  through  the  algorithm’s  metadata  analytics,  will  set  a  standard  for  adequate  due 
diligence  in  handling  the  challenging  budgetary  issues  related  to  delivering  reliably  on-time, 
within-schedule,  and  at  the  quality  required.  As  demonstrated  in  the  case  studies,  the  6-3-5 
Method  making  data  more  intelligent  starting  with  APRI  tagging  is  a  needed  solution  in 
today’s  “keep-up  or  fall-behind”  information-based  society  in  order  to  sustain  our  Navy 
operational  capabilities,  forward  presence  and  warfighting  advantage. 

Based  on  the  findings  contained  in  this  white  paper,  it  is  recommended  that  DoN 
leadership  evaluate  and  deploy  the  WA2BE  Algorithm  and  its  metadata  constructs, 
according  to  the  6-3-5  Method,  to  estimate  and  manage  expenditures  reliably,  while 
improving  the  quality  and  health  of  programs  in  support  of  existing  Joint  Capabilities 
Integration  and  Development  System  (JCIDS);  Planning,  Programming,  Budgeting  and 
Execution  (PPBE),  including  Program  Objective  Memorandum  (POM)  submission;  and 
Defense  Acquisition  System  (DAS)  processes,  including  principal  Major  Defense  Acquisition 
Programs  (MDAPS),  for  example,  the  Ohio-class  SSBN  replacement  and  naval  Joint  Strike 
Fighter  variants  (Secretary  of  the  Navy,  201 1 ). 
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When  combining  the  results  from  all  three  case  studies,  the  SWLIN/TPS  deliverable 
analysis  evidence  indicates  that  application  of  the  6-3-5  Method  would  improve  the  health  of 
Navy  Programs  of  Record  by  supporting  the  respective  Acquisition  Program  Management 
Offices  and  those  naval  warfare  systems’  centers  of  excellence  responsible  for  in-service 
engineering  and  life-cycle  sustainment  support.  The  case  studies  demonstrate  critical  gaps 
that  are  affecting  the  recapitalization  and  sustainment  of  the  world’s  finest  Navy.  This  paper 
demonstrates  how  the  6-3-5  Method  can  immediately,  effectively,  and  efficiently  satisfy 
these  DoD  and  DoN  needs,  addressing  these  critical  gaps  through  use  a  metadata-based, 
statistics  and  heuristic  assessment  algorithm  resulting  in  improved  cost  control  capability. 
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Percent  where  Statistical  Confidence  Analysis 
was  Greater  than  80%  for  RCOH  Shop  Bids 


41%  59% 


Bid  Needs  to  Be  within  10%  of  Estimate  (RCOH  Contract  Requirement) 

Percent  Over  Bid 


Percent  where  Statistical  Confidence  Analysis 
was  Greater  than  80%  for  RCOH  Trade  Bids 
57%  41%  2% 


Bid  Needs  to  Be  within  10%  of  Estimate  (RCOH  Contract  Requirement) 

Percent  Over  Bid 
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PEO  CARRIERS 


CASE  STUDY  DEMONSTRATED... 


6-3-5 

Method 

Statistical  Algorithm 

Methods  &  Measurements 

Visual  Representation 

Assessing  Bid 
Offers  to  Find 
Cost  Variance 
Issues  using 
Metadata 


Resolving  Cost 
Variance  Issues 
Before  Contract 
Award  using 
Metadata 


Resolving  Cost 
Variance  Issues 
During  Contract 
Execution  using 
Metadata 


Example... 


Problem  Solving 
Collaboration 
Approach 


Promoting 
Acquisition  Team 
Due  Diligence 


Statistical  Confidence  Analysis  Target  Goal 

100%  (J) 

o%  o%  o%  o% 


0%  2.5%  7.5%  12.5%  17.5%  22.5% 

Bid  Needs  to  Be  within  10%  of  Estimate  (RCOH  Contract  Requirement) 

Percent  Over  Bid 
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NATO  PROGRAM  CASE  STUDY  DEMONSTRATED. 


6-3-5 

Method 

Statistical  Algorithm 

Methods  &  Measurements 

Visual  Representation 

Assessing  Bid 
Offers  to  Find 
Cost  Variance 
Issues  using 
Metadata 


Resolving  Cost 
Variance  Issues 
Before  Contract 
Award  using 
Metadata 


© 


Resolving  Cost 
Variance  Issues 
During  Contract 
Execution  using 
Metadata 


Initial  Workforce  Alignment  Assessment 

>20% 

- 1 - 


<  80% 

JL 


Estimate 


K 


Grades: 


/o\ 

/e\ 

°/ 

0%  2,5%  7.5%  12.5%  17.5%  22.5% 

Bid  Needs  to  Be  within  10%  of  Estimate  (RCOH  Contract  Requirement) 

Percent  Over  Estimate 


Problem  Solving 
Collaboration 
Approach 


Promoting 
Workforce-Driven 
Innovation  Due 
Diligence 


Air  Traffic  Control 
System  (HW  &  SW) 
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After  Workforce  Innovation  Due  Diligence 

>80% 

- 1 - 


<  20% 

J 


Estimate 


Grades: 

/o\ 

6\ 

/f\ 

o) 

/  ©  N 

o 

0%  2.5%  7.5%  12.5%  17.5%  22.5% 

Bid  Needs  to  Be  within  10%  of  Estimate  (RCOH  Contract  Requirement) 

Percent  Over  Estimate 
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NAVSEA  CENTERS  CASE  STUDY  DEMONSTRATED. 


6-3-5 

Method 

Statistical  Algorithm 

Methods  &  Measurements 

Visual  Representation 

Assessing  Bid 
Offers  to  Find 
Cost  Variance 
Issues  using 
Metadata 


Resolving  Cost 
Variance  Issues 
Before  Contract 
Award  using 
Metadata 


s> 


Resolving  Cost 
Variance  Issues 
During  Contract 
Execution  using 
Metadata 


Meets 

Quality 

Needs 


Percent  where  Statistical  Confidence 
Analysis  was  Greater  than  80%  for 
Quality  Use  of  Appropriations 


49% 

JL 


9% 

JL 


r^r 


39% 

JL 


i 

1 

Grades:  j 

A 

© 

A 

0/ 

P 

A 

•  r 

I  '  / 

\  / 

Y 

v 

3%T8D 

Good  Quality 

Marginal  | 

1 

Poor  Quality  ^ 

Work  Quality 


Multiple  Branches 


Problem  Solving 
Collaboration 
Approach 


Promoting 
Workforce- Driven 
Innovation  Due 
Diligence 


Meets  percent  where  Statistical  Confidence 
Quality 

Needs 


Analysis  was  Greater  than  80%  after 
Workforce  Due  Diligence 


65% 

JL 


9% 

rAr 


23% 

JL 


i 

i 

Grades:  j 

1  / 
1  / 

• 

© 

_ 

A 

°/ 

(o) 

j  y 

y 

A 

3%  TBD 

Good  Quality 

Marginal  | 

1 

Poor  Quality  ^ 

Work  Quality 


Two  Centers 
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THE  6-3-5  METHOD  IN  SUPPORT  OF 
DON  POM  SUBMISSION  PROCESS 


Provides  "A  Priori"  Support  of 
DON  POM  Submission  Process 

Statistically  Same  or 


Similar  Performance  Data 

v 


Statistical  Confidence  of  80% 
or  greater  in  Estimate 

0  Within  7.5% 

$  Within  7.5%  to  12.5% 

Greater  Than  12.5% 


FYDP  for  Fixed  Wing  Aircraft 


Reliability  of  FYDP  and  Visibility  of  Risks 


Statistical  Algorithm 


Methods  &  Measurements 


Visual  Representation 


Assessing  Bid 
Offers  to  Find 
Cost  Variance 
Issues  using 
Metadata 


Resolving  Cost 
Variance  Issues 
Before  Contract 
Award  using 
Metadata 

Resolving  Cost 
Variance  Issues 
During  Contract 
Execution  using 
Metadata 
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